Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structured metadata: Refactor into new variable #826

Merged
merged 73 commits into from
Oct 21, 2024

Conversation

gtk-grafana
Copy link
Contributor

@gtk-grafana gtk-grafana commented Oct 15, 2024

This PR moves structured metadata fields into a dedicated scenes variable.
This enables us to have more control over how metadatafields are interpolated in the query, allowing us to place all metadata before any log parsers, which will result in more efficient/optimal queries.

There should be no changes to the UI, just the queries that are sent to Loki.

We still have a potential race condition where users can filter results in the table/logs panel before the detected_fields response which will give us the parser, in these cases we will fall back to adding fields with "unknown" parsers to the fields variable, and require both logfmt and json parsers for queries with that field. Created #837 to track

gtk-grafana and others added 30 commits September 23, 2024 15:09
@gtk-grafana gtk-grafana self-assigned this Oct 16, 2024
@gtk-grafana gtk-grafana added the enhancement New feature or request label Oct 16, 2024
@gtk-grafana gtk-grafana marked this pull request as ready for review October 17, 2024 16:23
@gtk-grafana gtk-grafana requested a review from a team as a code owner October 17, 2024 16:23
@gtk-grafana gtk-grafana added this to the 1.0.2 milestone Oct 17, 2024
key: string,
value: string,
sceneRef: SceneObject
): VariableFilterType {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's something very odd with this function. type can be null, or have a value; independent of the type, we check for getParserForField(), which supports only structuredMetadata, so it can return something else, or nothing, but type can also be null, which goes to the default case.

I think we should refactor this before merging.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion:

  • Make type non-null
  • In the consumer side of these functions, if there is no type to pass, then use getParserForField(), or other function, instead of getFilterTypeFromLabelType().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I fully understood, how does this look? b643f48

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better, thanks.

Copy link
Contributor

@matyax matyax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just one comment.

Copy link
Contributor

@svennergr svennergr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great job!

Just a nit, might also be interesting for @matyax:
image

When filtering for structured metadata the logs panel filter button state is not updated accordingly. Maybe Matias knows best where to adjust that.

@matyax
Copy link
Contributor

matyax commented Oct 21, 2024

Taking a look.

@matyax
Copy link
Contributor

matyax commented Oct 21, 2024

@gtk-grafana
Copy link
Contributor Author

@svennergr @matyax Good catch, that bug has likely been hanging out since we added detected_fields. Should be fixed now in b643f48

@matyax
Copy link
Contributor

matyax commented Oct 21, 2024

I pulled the latest changes and still don't see the active state:

imagen

Copy link
Contributor

@matyax matyax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢 🇮🇹

@gtk-grafana gtk-grafana merged commit 0d5ed56 into main Oct 21, 2024
4 checks passed
@gtk-grafana gtk-grafana deleted the gtk-grafana/structured-metadata-refactor branch October 21, 2024 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants